路由器先验偏置：在 MoE 后训练中保持基础路由结构

文章背景与核心概要

混合专家（MoE）模型在预训练阶段通常采用辅助负载均衡损失（LBL）来鼓励专家利用率的均匀分布。然而，在后训练阶段，基础路由器已经编码了专门的、非均匀的专家协同激活结构。重新施加均匀性目标会抹平这些至关重要的结构，而强制执行硬约束则会导致性能下降。

本文引入了路由器先验偏置（Router Prior Bias, RPB）——这是一种软路由器锚定技术，它将路由器逻辑值（logits）拉向源自冻结基础路由器的先验，同时保持路由器本身的可训练性。实验表明，与传统的负载均衡损失和无锚定微调相比，RPB 显著提高了域内和域外的性能。

摘要 (Abstract)

Mixture-of-Experts (MoE) pretraining relies on an auxiliary load-balancing loss (LBL) to drive per-expert utilization toward uniformity. Post-training inherits a different situation: the base router already encodes non-uniform expert co-activation structure, which a re-imposed uniformity objective flattens away.

混合专家（MoE）预训练依赖于辅助负载均衡损失（LBL）来推动每个专家的利用率趋向均匀。后训练则继承了不同的情况：基础路由器已经编码了非均匀的专家协同激活结构，而重新施加的均匀性目标会将其抹平。

We show that downstream performance depends instead on holding this inherited routing softly, a principle we term soft router anchoring, and instantiate it as Router Prior Bias (RPB), a training-time bias that pulls the router logits toward a prior read off the frozen base router while leaving the router itself trainable.

我们表明，下游性能实际上取决于对这种继承路由的温和保持，我们将这一原则称为软路由器锚定（soft router anchoring），并将其实例化为路由器先验偏置（Router Prior Bias, RPB）。这是一种训练时的偏置，它将路由器逻辑值拉向从冻结的基础路由器中读取的先验，同时保持路由器本身可训练。

Performance Highlights:

On math post-training of Moonlight-16B-A3B, RPB achieves 45.77 in-domain accuracy compared to 31.91 under re-applied LBL and 29.44 under unanchored fine-tuning, while better retaining out-of-domain capabilities.

The performance advantage over LBL successfully reproduces across model families (e.g., Qwen3-30B-A3B-Base).

性能亮点：
在对 Moonlight-16B-A3B 进行数学后训练时，RPB 实现了 45.77 的域内准确率，而重新应用 LBL 时为 31.91，无锚定微调时为 29.44，同时更好地保持了域外能力。
相比 LBL 的性能优势已成功在不同的模型系列（例如 Qwen3-30B-A3B-Base）中复现。

Key Insights:

Anchors defined on router weights, logits, or output distributions perform comparably, indicating that the core benefit stems from the softness of the constraint rather than the specific prior provided by RPB.

Preserving community structure in the expert co-activation graph tracks these gains. However, enforcing the same prior as a hard assignment causes performance to drop sharply.

Therefore, community structure is a footprint of soft anchoring rather than its direct source—demonstrating that inherited routing should be held softly during post-training to avoid the costs of both flattening and rigid enforcement.

核心见解：
定义在路由器权重、逻辑值或输出分布上的锚定表现相当，这表明核心收益来自于约束的软性（softness of the constraint），而不是 RPB 提供的特定先验。
保持专家协同激活图中的社群结构与这些性能增益密切相关。然而，将相同的先验作为硬分配强制执行会导致性能急剧下降。
因此，社群结构是软锚定的足迹，而不是其直接来源——这证明了在后训练期间应温和地保持继承的路由，以避免平坦化和刚性执行带来的双重代价。